Method and system for identifying exceptions of people behavior

ABSTRACT

A method and system for identifying exceptions, of a person/child behavior through scheduled behavior, using a personal communication device having at least one sensor attached to person/child body which provide location data and motion data. The method includes sampling measurements data for each sensor of the personal communication device, identifying atomic pattern of sampled measurements, the atomic pattern representing basic behavior of the person/child, identify location in which person has spent time using a clustering algorithm, create activity segmentation characterized by location, schedule, caregiver, or content derived from sensors measurements and atomic patterns and analyzing characteristics changes or combination thereof of activities in sequenced/complex activities in comparison to baseline, to identify exception which indicate of at least one of the following: changes at activity level, changes at emotional arousal level, unknown locations or unexpected locations based on schedule.

FIELD OF THE INVENTION

The present invention relates generally to the field of systems andmethods for tracking, analyzing, and identifying exceptions in subjectbehavior and more particularly, to systems and methods for tracking,analyzing, and identifying exceptions in subject behavior throughscheduled location based behavior.

BACKGROUND OF THE INVENTION

People with disabilities and in particular, those with communication andcognitive disabilities may experience great difficulties to expresstheir feelings and emotions. The professional literature indicates thatpeople with autism or other communication and cognitive disabilities arevulnerable and subjected to mistreatment and higher rates of violenceand abuse (as compared to their peers without a disability). Hence,there is a need to ensure their personal safety and to develop toolsthat will allow their family members and service providers to have abetter sense of their well-being.

SUMMARY OF THE INVENTION

The present invention provides a method for identifying exceptions, of aperson behavior through scheduled behavior, using a personalcommunication device associated with least one sensor attached to aperson body or garments which provides at least location data and/ormotion data. The method comprising the steps of: sampling measurementsdata for each sensor associated with the person communication device,identifying atomic pattern of sampled measurements, said atomic patternrepresenting basic behavior of the person, identify locations in whichthe person has spent time using a clustering algorithm, create activitysegmentation characterized by location, schedule, activity type, orcontent derived from sensors measurements and atomic patterns andanalyzing characteristics changes or combination thereof of activitiesin sequenced/complex activities in comparison to given baseline, toidentify exception which indicate of at least one of the following:changes at activity level, changes at emotional arousal level, unknownlocations or unexpected locations based on schedule.

According to some embodiments of the present invention the methodfurther comprising the step of creating visualization presentation ofthe scheduled location based on activity segmentation.

According to some embodiments of the present invention the methodfurther comprising the steps of receiving sensor measurements receivedfrom communication device associated with a caregiver or person/childcurrently located in vicinity of the person/child and identifyingcorrelations/association between different sensors measurements.

According to some embodiments of the present invention the methodfurther comprising the steps of receiving data from environmentalsensors.

According to some embodiments of the present invention the methodfurther comprising the steps of receiving location based data from webbased or external services.

According to some embodiments of the present invention the methodfurther comprising the step of creating a hierarchical representation ofactivity sequences of time period with the same context.

According to some embodiments of the present invention the methodfurther comprising the step of using schedule information provided bythe guardian or other external sources to refine segmentation.

According to some embodiments of the present invention the methodcomprising the step of receiving information from guardian associating aspecific pattern or a specific time interval with a specific emotion andapplying algorithms to learn the typical behavior of the personrepresenting each emotion using the information provided by theguardian.

According to some embodiments of the present invention the method thefurther comprising the step of using predefined domain specific rulesfor determining a reporting mechanism for sending an alert of behaviorexceptions based on at least one of the following: severity ofexceptions, classifications determined in the supervised learning stageand user preferences.

According to some embodiments of the present invention the methodfurther comprising the step of providing the guardian with communicationinterface to perform inquiries based on exceptions.

According to some embodiments of the present invention the activitiessegmentation include classifying transits state between locations.

The present invention provides a system for identifying exceptions, of aperson behavior through scheduled behavior. The system comprised of: apersonal communication device having at least one sensor attached toperson body which provide location data and/or motion data, saidcommunication associated with a sensor processing module for samplingmeasurements data for each sensor of the personal communication deviceand identifying atomic pattern of sampled measurements, said atomicpattern representing basic behavior of the person and a sever includingsegmentation modules for identifying location in which the person hasspent time using a clustering algorithm, creating activity segmentationcharacterized by location, schedule, caregiver, or content derived fromsensors measurements and atomic patterns and Exception Identificationmodule for analyzing characteristics changes or combination thereof ofactivities in sequenced/complex activities in comparison to givenbaseline, to identify exception which indicate of at least one of thefollowing: changes at activity level, changes at emotional arousallevel, unknown locations or unexpected locations based on schedule.

According to some embodiments of the present invention the systemfurther comprising visualization presentation of the scheduled locationbased on activity segmentation.

According to some embodiments of the present invention the segmentationmodule further receives sensor measurements received from communicationdevice associated with a caregiver or person currently located invicinity of the person and identifies correlations/association betweendifferent sensors measurements.

According to some embodiments of the present invention the segmentationmodules further receives data from environmental sensors.

According to some embodiments of the present invention the segmentationmodule further receives location based data from web based services.

According to some embodiments of the present invention the segmentationmodule further creates a hierarchical representation of activitysequences of time period with the same context.

According to some embodiments of the present invention the segmentationmodule further uses schedule information provided by the guardian orother external sources to refine segmentation.

According to some embodiments of the present invention the ExceptionIdentification module further receives information from guardianassociating a specific pattern or a specific time interval with aspecific emotion and applying algorithms to learn the typical behaviorof the person representing each emotion using the information providedby the guardian.

According to some embodiments of the present invention the ExceptionIdentification module further uses predefined domain specific rules fordetermining a reporting mechanism for sending an alert of behaviorexceptions based on at least one of the following: severity ofexceptions, classifications determined in the supervised learning stageand user preferences.

According to some embodiments of the present invention the systemfurther comprising communication interface for the guardian to performinquiries based on exceptions. According to some embodiments of thepresent invention the segmentation module include classifying transitsstate between locations.

BRIEF DESCRIPTIONS OF THE DRAWINGS

The subject matter regarded as the invention will become more clearlyunderstood in light of the ensuing description of embodiments herein,given by way of example and for purposes of illustrative discussion ofthe present invention only, with reference to the accompanying drawings,wherein

FIG. 1 is a block diagram, schematically illustrating a system fortracking, analyzing, and identifying exceptions in subject behavior,according to some embodiments of the invention;

FIG. 2 is a flowchart, schematically illustrating a method for sensorsprocessing, according to some embodiments of the invention;

FIG. 3 is a flowchart, schematically illustrating a method of PowerManagement, according to some embodiments of the invention;

FIG. 4 is a flowchart, schematically illustrating a method for DaySegmentation by Location, according to some embodiments of theinvention;

FIG. 5 is a flowchart, schematically illustrating a method for DaySegmentation by Activities, according to some embodiments of theinvention;

FIG. 6 is a flowchart, schematically illustrating a method forSupervised Learning of Behaviors Classification by Emotion, according tosome embodiments of the invention;

FIG. 7 is a flowchart, schematically illustrating a method for ExceptionIdentification and Reporting, according to some embodiments of theinvention;

FIG. 8 is a flowchart, schematically illustrating a Person Applicationprocessing, according to some embodiments of the invention; and

FIG. 9 is a flowchart, schematically illustrating Caregiver Applicationprocessing, according to some embodiments of the invention.

DETAILED DESCRIPTIONS OF SOME EMBODIMENTS OF THE INVENTION

While the description below contains many specifications, these shouldnot be construed as limitations on the scope of the invention, butrather as exemplifications of the preferred embodiments. Those skilledin the art will envision other possible variations that are within itsscope. Accordingly, the scope of the invention should be determined notby the embodiment illustrated, but by the appended claims and theirlegal equivalents.

An embodiment is an example or implementation of the inventions. Thevarious appearances of “one embodiment,” “an embodiment” or “someembodiments” do not necessarily all refer to the same embodiments.Although various features of the invention may be described in thecontext of a single embodiment, the features may also be providedseparately or in any suitable combination. Conversely, although theinvention may be described herein in the context of separate embodimentsfor clarity, the invention may also be implemented in a singleembodiment.

Reference in the specification to “one embodiment”, “an embodiment”,“some embodiments” or “other embodiments” means that a particularfeature, structure, or characteristic described in connection with theembodiments is included in at least one embodiments, but not necessarilyall embodiments, of the inventions. It is understood that thephraseology and terminology employed herein is not to be construed aslimiting and are for descriptive purpose only.

The principles and uses of the teachings of the present invention may bebetter understood with reference to the accompanying description,figures and examples. It is to be understood that the details set forthherein do not construe a limitation to an application of the invention.Furthermore, it is to be understood that the invention can be carriedout or practiced in various ways and that the invention can beimplemented in embodiments other than the ones outlined in thedescription below.

It is to be understood that the terms “including”, “comprising”,“consisting” and grammatical variants thereof do not preclude theaddition of one or more components, features, steps, or integers orgroups thereof and that the terms are to be construed as specifyingcomponents, features, steps or integers. The phrase “consistingessentially of”, and grammatical variants thereof, when used herein isnot to be construed as excluding additional components, steps, features,integers or groups thereof but rather that the additional features,integers, steps, components or groups thereof do not materially alterthe basic and novel characteristics of the claimed composition, deviceor method.

If the specification or claims refer to “an additional” element, thatdoes not preclude there being more than one of the additional element.It is to be understood that where the claims or specification refer to“a” or “an” element, such reference is not be construed that there isonly one of that element. It is to be understood that where thespecification states that a component, feature, structure, orcharacteristic “may”, “might”, “can” or “could” be included, thatparticular component, feature, structure, or characteristic is notrequired to be included.

Where applicable, although state diagrams, flow diagrams or both may beused to describe embodiments, the invention is not limited to thosediagrams or to the corresponding descriptions. For example, flow neednot move through each illustrated box or state, or in exactly the sameorder as illustrated and described.

Methods of the present invention may be implemented by performing orcompleting manually, automatically, or a combination thereof, selectedsteps or tasks. The term “method” refers to manners, means, techniquesand procedures for accomplishing a given task including, but not limitedto, those manners, means, techniques and procedures either known to, orreadily developed from known manners, means, techniques and proceduresby practitioners of the art to which the invention belongs. Thedescriptions, examples, methods and materials presented in the claimsand the specification are not to be construed as limiting but rather asillustrative only.

Meanings of technical and scientific terms used herein are to becommonly understood as by one of ordinary skill in the art to which theinvention belongs, unless otherwise defined. The present invention canbe implemented in the testing or practice with methods and materialsequivalent or similar to those described herein.

Any publications, including patents, patent applications and articles,referenced or mentioned in this specification are herein incorporated intheir entirety into the specification, to the same extent as if eachindividual publication was specifically and individually indicated to beincorporated herein. In addition, citation or identification of anyreference in the description of some embodiments of the invention shallnot be construed as an admission that such reference is available asprior art to the present invention.

FIG. 1 is a block diagram, schematically illustrating a system fortracking, analyzing, and identifying exceptions in subject behavior,according to some embodiments of the invention. The system according tosome embodiments of the present invention is comprised of at least one aPersonal Tracking Device 10 for measuring behavior data of a Person, aserver 50 for analyzing measured data and identifying exceptionalbehavior based on transmitted measured data, and Person database 80 formaintaining history of measured data and analyzed data. The system maybe further be associated with one of the following information sources:Caregiver web based Application 20 for receiving location data andreports, Guardian web based Application 30, environmental Sensors 40enabling to measure environment condition in vicinity of the Person,such as temperature or monitoring the environment using camera ormicrophone, location-based Web Services 60 for providing real-time dataat the current position of the Person and/or personal applicationenabling to track in real-time the person behavior and provide feedback.The Personal Tracking Device 10 basic structure includes at least one ofthe following: location sensor such as GPS, motion sensor such anaccelerator, and or microphone. Optionally the Personal Tracking Device10 includes sensor processing module 100 for sampling, compressing,filtering and/or power management module 110. The server 50 maycomprise: Sensor Processing 510 for analyzing atomic behavior pattern,Day Segmentation by Location 520 and Day Segmentation by Activities 530for analyzing and identifying activities and their characteristics,Supervised Learning of Behaviors Classification by Emotion 540 enablingto classify behaviors using guardian or caregivers feedback andException Identification and Reporting module 550 for identifyingexceptional behavior based on identified activities and theircharacteristics by comparing to predefined base line. The powermanagement and sensor processing modules may be processed at thetracking device or at the server or in both locations.

According to some embodiments of the present invention the server mayinclude management module for automatically controlling the personaltracking device sensors and modes of operation: such as listen-in modeor to ‘ring’ mode or to ‘bi-directional call’ mode or out of deep sleepmode and optionally defining settings of the device such as authorizednumbers for listen-in or automatically setup geo-fences parameters.

FIG. 2 is a flowchart, schematically illustrating a method for sensorsprocessing 100, according to some embodiments of the invention. Themethod includes: at least one of the following operations: Smartactivation of sensors based on data from other sensors, (E.g. activatemicrophone when measuring extreme forces in the accelerometer 1020),Collecting data from different sensors 1030, Data sampling, compression,filtering noise reduction and filtering 1040 and or identification ofatomic patterns such as location change descriptions, motion patternsand keywords 1050.

The sensor of the tracking device may include at least one of thefollowing: GPS, G-Sensor, Gyro, P-Sensor, Microphone, Camera, Heart BeatMonitor, blood pressure and GSR (Galvanic Skin Sensor).

The compression and filter processes may include one of the followingactions:

For GPS sensor, accumulating of readings and sending measurements atchanging frequencies based on activity, eliminating redundant GPSlocations (same location) and incremental representation of the GPSreadings (send an initial full GPS location and then only thedifferences), optionally transmitting only meaningful atomic motionpatterns and not the raw data.

Acceleration sensor: sampling of specific actual movements such as steps(pedometer) hand flapping, head banging, stretching skin etc. as well assome overall summary characteristics of movement (e.g. an indication oflevel of activity per minute).

Microphone eliminating data below a certain volume and eliminating datawhere no human sound was detected, identifying keywords.

Noise reduction may include, implementing noise reduction such as, GPSspikes elimination, smoothing and noise reduction can be done over theraw acceleration samples (imply unreasonable speed).

For Arousal sensors: sending only significant readings representingarousal over a certain level and/or sending summary data. E.g. averagesreadings over a number of seconds, heartbeat variability over a numberof seconds, etc.

Identification of atomic patterns is performed by analyzing real timebasic behavior pattern, comparing to known pattern templates ofclassified action such as walking, hand clapping, etc.

FIG. 3 is a flowchart, schematically illustrating a method of PowerManagement 200, according to some embodiments of the invention. Thismethod may include one of following steps: Identifying indoor locationusing GPS, CELL ID information as well as location information ofnetwork communication elements such as Wi-Fi or Bluetooth transmitters,to reduce GPS data transmission frequency 2020 and/or Set a dynamicgeo-fence at the device, based on the learnt locations in the server toimmediately identify exiting the location 2030.

The system according to some embodiments of the present invention useslearning algorithms to detect information regarding locations in whichthe person/child spends time in. The location information may includeparameters such as the boundaries of the location for defining ageo-fence alert.

Optionally the system of the present invention uses supervised data fromthe guardian or caregiver for providing meaningful names/tags toidentified locations.

Once identifying a child/person, entering a location which is identifiedby name, the server automatically uses the history data aggregated bythe learning algorithm related to this location for dynamically settinga geo-fence range which can be used at the personal device to generatealerts. The personal device may send GPS data once identifying theperson/child is out of the geo-fence range.

The geo-fence range may be dependent on multiple parameters, includingthe aggregated history of GPS data or other communication element (suchas Bluetooth) accumulated by the learning algorithm and the history ofGPS errors.

FIG. 4 is a flowchart, schematically illustrating a method for DaySegmentation by Location 520, according to some embodiments of theinvention. This method may include one of following steps: Receivingreal-time location sensor data of multiple sensors associated with aperson carrying personal tracking device and caregiver including: GPS,acceleration, gyro, WIFI, Bluetooth, cellular network, orientationsensor 5210, analyzing location data across different sources to reducelocation error and improve its accuracy 5220, Identify location in whichthe Person has spent time using a clustering algorithm 5225, (thelocation can be identified by type such as school, park, home and/or byspecific names of places), Classifying transits state between locationssuch drives, walks, etc. 5230, and create an hierarchical representationof location data by context, e.g. grouping transits and stops to theentire drive from home to school 5240.

The analysis of location data include checking caregiver location andlocation of other Persons that are expected to be in same location. Theanalysis may applied by using techniques such as Kalman Filter.

According to some embodiments of the present invention, the location canbe refined by associating the location data to the schedule of thePersons based on given schedule by the guardian, other external sourcesor learned scheduled 5250.

FIG. 5 is a flowchart, schematically illustrating a method for DaySegmentation by Activities 530, according to some embodiments of theinvention. This method may include one of following steps: Identifyingcorrelations from different sensors of person/child, guardian caregiverand/or environmental sensors 5310, Receiving, identified atomic patternsand activities characteristics in real-time 5320, Creating activitysegmentation, where an activity is identified by analyzing schedule,location, and/or caregiver and/or content of activity 5330 and/orCreating a hierarchical representation of activity sequences 5340.

According to some embodiments of the present invention the segmentationis achieved by analyzing sampled raw measurement data and atomicpatterns.

The hierarchical representation may include a sequence of activities oftime period with the same context, for example the entire school staythat is composed out of the different activities in school or the entireride from home to school that is composed of the stops and transitsbetween them.

FIG. 6 is a flowchart, schematically illustrating a method forSupervised Learning of Behaviors Classification by Emotion 540,according to some embodiments of the invention. This method includes atleast one of the following steps: Receiving information from guardianassociating a specific pattern or a specific time interval with aspecific emotion 5410 and/or Applying algorithms to learn the typicalbehavior of the person/child representing each emotion using theinformation provided by the parent 5420. The received information mayinclude, indicating that a head banging identifies stress, or that theentire last 5 minutes represent stressful behavior.

FIG. 7 is a flowchart, schematically illustrating a method for ExceptionIdentification and Reporting 550, according to some embodiments of theinvention. The method includes at least one of the following steps:Analyzing characteristics changes and/or irregular activities insequence/complex activities in all level of activities hierarchies incomparison to (predefined rules/templates) baseline 5510, usingpredefined domain specific rules for determining reporting mechanism5520, and allow guardian to perform inquiries based on exceptions 5530,e.g. opening microphone or camera remotely for verifying person/child'sstatus/reaction and optionally communicate with the person/child.

The characteristics analysis enable to identify exceptions such as:changes at activity level, changes at emotional arousal level, unknownlocations or unexpected locations based on schedule (such as SuspectedAbuse in Transportation Event) longer transit routes, different transitroutes. The exception analysis may be performed using time seriesanalysis techniques.

The reporting mechanism may include at least one of the following: dailyreport, weekly report, real-time notification usingemail/SMS/Notifications.

The predefined domain specific rules may be based on at least one of thefollowing: severity of exceptions, classifications determined in thesupervised learning stage and user preferences.

FIG. 8 is a flowchart, schematically illustrating a Person Applicationprocessing 70, according to some embodiments of the invention. Theperson application may perform at one of the following: Generate avisual representation of the day scheduled segments by location or byactivities, using pictures of locations or activities retrieved from webdatabases as well as pictures taken by caregiver or guardian 710, enableguardian to collect feedback from the person 720 and or potentially addperson/child's reaction classification by parent into the activityanalysis 730.

FIG. 9 is a flowchart, schematically illustrating Caregiver Applicationprocessing 80, according to some embodiments of the invention. TheCaregiver application may perform the following: Allow caregivers toreport on the person activities (e.g. eating, medical taking, physicalactivities) input classifications of Person state (e.g. stress level)810 and/or Analyze and correlate caregivers classifications toactivities to identify e.g. stressful activities 820.

The invention claimed is:
 1. A method of remotely monitoring theemotional state of a cognitively disabled patient, using a personalcommunications computing device associated with at least a location datasensor, a motion data sensor, and a microphone attached to the patient'sbody or garments, said method comprising the steps of: samplingmeasurements from the location and motion data sensors; identifyingatomic patterns of the sampled motion measurements and identifying atime and a geographic location in which the atomic patterns are detectedby analyzing real time basic behavior pattern and comparing theidentified time and geographic location of the atomic pattern to knownpattern templates of classified action; receiving and storing in acomputer readable storage medium information indicative of typicalbehavior from a guardian associating at least one specific atomicpattern or at least one specific time interval with at least onespecific emotion of the cognitively disabled patient, the receivedinformation identifying stress from the atomic pattern or indicating thespecific interval represents stressful behavior; using a clusteringalgorithm generated by computer-implemented supervised learning toclassify activity segments in real time characterized by the location,the time, and the stored information received from the guardian, theclassification resulting in at least a first activity segmentcharacterized by a first location and a second activity segmentcharacterized by a second location, the first and second locations beingdifferent from one another; comparing at least one of characteristicschanges and irregular activities in the classified activity segmentswith a baseline to identify an exception indicating an emotional state;responsively to identifying the exception, activating the microphone andsampling input from the microphone to identify one or more keywordsrelated to the cognitively disabled patient provided by the guardian;and responsively to the identification of the one or more keywords,reporting an emotional arousal alert.
 2. The method of claim 1 furthercomprising the step of sending an alert indicating a severity of theexception.
 3. The method of claim 1 wherein the activity segmentationfurther includes classifying transitions between locations.
 4. A systemfor remotely monitoring the emotional state of a cognitively disabledpatient, the system comprising a personal communications computingdevice associated with at least a location data sensor, a motion datasensor, and a microphone attached to the patient's body or garments, thesystem configured to perform the steps of: sampling measurements fromthe location and motion data sensors; identifying atomic patterns of thesampled motion measurements and identifying a time and a geographiclocation in which the atomic patterns are detected by analyzing areal-time basic behavior pattern and comparing the identified time andgeographic location of the atomic pattern to known pattern templates ofclassified actions; receiving and storing in a computer readable storagemedium information indicative of typical behavior from a guardianassociating at least one specific atomic pattern or at least onespecific time interval with at least one specific emotion of thecognitively disabled patient, the received information identifyingstress from the atomic pattern or indicating the specific intervalrepresents stressful behavior; using a clustering algorithm generated bycomputer-implemented supervised learning to classify activity segmentsin real time characterized by the location, the time, and the storedinformation received from the guardian, the classification resulting inat least a first activity segment characterized by a first location anda second activity segment characterized by a second location, the firstand second locations being different from one another; comparing atleast one of characteristics changes and irregular activities in theclassified activity segments with a baseline to identify an exceptionindicating an emotional state; responsively to identifying theexception, activating the microphone and sampling input from themicrophone to identify one or more keywords related to the cognitivelydisabled patient provided by the guardian; and responsively to theidentification of the one or more keywords, reporting an emotionalarousal alert.
 5. The system of claim 4, wherein the personalcommunications computing device is further configured for sending analert indicating a severity of the exceptions.